56 research outputs found
Image Processing and Machine Learning for Hyperspectral Unmixing: An Overview and the HySUPP Python Package
Spectral pixels are often a mixture of the pure spectra of the materials,
called endmembers, due to the low spatial resolution of hyperspectral sensors,
double scattering, and intimate mixtures of materials in the scenes. Unmixing
estimates the fractional abundances of the endmembers within the pixel.
Depending on the prior knowledge of endmembers, linear unmixing can be divided
into three main groups: supervised, semi-supervised, and unsupervised (blind)
linear unmixing. Advances in Image processing and machine learning
substantially affected unmixing. This paper provides an overview of advanced
and conventional unmixing approaches. Additionally, we draw a critical
comparison between advanced and conventional techniques from the three
categories. We compare the performance of the unmixing techniques on three
simulated and two real datasets. The experimental results reveal the advantages
of different unmixing categories for different unmixing scenarios. Moreover, we
provide an open-source Python-based package available at
https://github.com/BehnoodRasti/HySUPP to reproduce the results
Fusion of Hyperspectral and LiDAR Data Using Sparse and Low-Rank Component Analysis
The availability of diverse data captured over the same region makes it possible to develop multisensor data fusion techniques to further improve the discrimination ability of classifiers. In this paper, a new sparse and low-rank technique is proposed for the fusion of hyperspectral and light detection and ranging (LiDAR)-derived features. The proposed fusion technique consists of two main steps. First, extinction profiles are used to extract spatial and elevation information from hyperspectral and LiDAR data, respectively. Then, the sparse and low-rank technique is utilized to estimate the low-rank fused features from the extracted ones that are eventually used to produce a final classification map. The proposed approach is evaluated over an urban data set captured over Houston, USA, and a rural one captured over Trento, Italy. Experimental results confirm that the proposed fusion technique outperforms the other techniques used in the experiments based on the classification accuracies obtained by random forest and support vector machine classifiers. Moreover, the proposed approach can effectively classify joint LiDAR and hyperspectral data in an ill-posed situation when only a limited number of training samples are available
Hyde: The First Open-Source, Python-Based, Gpu-Accelerated Hyperspectral Denoising Package
As with any physical instrument, hyperspectral cameras induce different kinds
of noise in the acquired data. Therefore, Hyperspectral denoising is a crucial
step for analyzing hyperspectral images (HSIs). Conventional computational
methods rarely use GPUs to improve efficiency and are not fully open-source.
Alternatively, deep learning-based methods are often open-source and use GPUs,
but their training and utilization for real-world applications remain
non-trivial for many researchers. Consequently, we propose HyDe: the first
open-source, GPU-accelerated Python-based, hyperspectral image denoising
toolbox, which aims to provide a large set of methods with an easy-to-use
environment. HyDe includes a variety of methods ranging from low-rank
wavelet-based methods to deep neural network (DNN) models. HyDe's interface
dramatically improves the interoperability of these methods and the performance
of the underlying functions. In fact, these methods maintain similar HSI
denoising performance to their original implementations while consuming nearly
ten times less energy. Furthermore, we present a method for training DNNs for
denoising HSIs which are not spatially related to the training dataset, i.e.,
training on ground-level HSIs for denoising HSIs with other perspectives
including airborne, drone-borne, and space-borne. To utilize the trained DNNs,
we show a sliding window method to effectively denoise HSIs which would
otherwise require more than 40 GB. The package can be found at:
\url{https://github.com/Helmholtz-AI-Energy/HyDe}.Comment: 5 page
Deep Hyperspectral Unmixing using Transformer Network
Currently, this paper is under review in IEEE. Transformers have intrigued
the vision research community with their state-of-the-art performance in
natural language processing. With their superior performance, transformers have
found their way in the field of hyperspectral image classification and achieved
promising results. In this article, we harness the power of transformers to
conquer the task of hyperspectral unmixing and propose a novel deep unmixing
model with transformers. We aim to utilize the ability of transformers to
better capture the global feature dependencies in order to enhance the quality
of the endmember spectra and the abundance maps. The proposed model is a
combination of a convolutional autoencoder and a transformer. The hyperspectral
data is encoded by the convolutional encoder. The transformer captures
long-range dependencies between the representations derived from the encoder.
The data are reconstructed using a convolutional decoder. We applied the
proposed unmixing model to three widely used unmixing datasets, i.e., Samson,
Apex, and Washington DC mall and compared it with the state-of-the-art in terms
of root mean squared error and spectral angle distance. The source code for the
proposed model will be made publicly available at
\url{https://github.com/preetam22n/DeepTrans-HSU}.Comment: Currently, this paper is under review in IEE
Multimodal Fusion Transformer for Remote Sensing Image Classification
Vision transformer (ViT) has been trending in image classification tasks due
to its promising performance when compared to convolutional neural networks
(CNNs). As a result, many researchers have tried to incorporate ViT models in
hyperspectral image (HSI) classification tasks, but without achieving
satisfactory performance. To this paper, we introduce a new multimodal fusion
transformer (MFT) network for HSI land-cover classification, which utilizes
other sources of multimodal data in addition to HSI. Instead of using
conventional feature fusion techniques, other multimodal data are used as an
external classification (CLS) token in the transformer encoder, which helps
achieving better generalization. ViT and other similar transformer models use a
randomly initialized external classification token {and fail to generalize
well}. However, the use of a feature embedding derived from other sources of
multimodal data, such as light detection and ranging (LiDAR), offers the
potential to improve those models by means of a CLS. The concept of
tokenization is used in our work to generate CLS and HSI patch tokens, helping
to learn key features in a reduced feature space. We also introduce a new
attention mechanism for improving the exchange of information between HSI
tokens and the CLS (e.g., LiDAR) token. Extensive experiments are carried out
on widely used and benchmark datasets i.e., the University of Houston, Trento,
University of Southern Mississippi Gulfpark (MUUFL), and Augsburg. In the
results section, we compare the proposed MFT model with other state-of-the-art
transformer models, classical CNN models, as well as conventional classifiers.
The superior performance achieved by the proposed model is due to the use of
multimodal information as external classification tokens
Multisource and Multitemporal Data Fusion in Remote Sensing
The sharp and recent increase in the availability of data captured by
different sensors combined with their considerably heterogeneous natures poses
a serious challenge for the effective and efficient processing of remotely
sensed data. Such an increase in remote sensing and ancillary datasets,
however, opens up the possibility of utilizing multimodal datasets in a joint
manner to further improve the performance of the processing approaches with
respect to the application at hand. Multisource data fusion has, therefore,
received enormous attention from researchers worldwide for a wide variety of
applications. Moreover, thanks to the revisit capability of several spaceborne
sensors, the integration of the temporal information with the spatial and/or
spectral/backscattering information of the remotely sensed data is possible and
helps to move from a representation of 2D/3D data to 4D data structures, where
the time variable adds new information as well as challenges for the
information extraction algorithms. There are a huge number of research works
dedicated to multisource and multitemporal data fusion, but the methods for the
fusion of different modalities have expanded in different paths according to
each research community. This paper brings together the advances of multisource
and multitemporal data fusion approaches with respect to different research
communities and provides a thorough and discipline-specific starting point for
researchers at different levels (i.e., students, researchers, and senior
researchers) willing to conduct novel investigations on this challenging topic
by supplying sufficient detail and references
Feature fusion of hyperspectral and lidar data using extinction profiles and total variation
To improve the classification of hyperspectral images, this paper proposes an approach for multi-sensor data fusion of LiDAR and hyperspectral data using extinction profiles and Orthogonal Total Variation Component Analysis (OTVCA). Results on the benchmark Houston data indicate the superior performance of the proposed approach compared to other approaches used in the experiments based on classification accuracies
- …